This document is versioned in a public git repo: https://github.com/cgranell/apps-mental-disorders-jmir.

This R notebook contains the code to produce the final figures and tables of the paper.

Data

The final number of surveyed papers is 158. For each paper, we have extracted 70 items. Each item is described in the methods section of the paper. About 71.5% (N=113) of papers reported some kind of assessment, whereas 28.5% (N=45) reported no assessment at all. Looking closer at the type of assessment: 13.9% (N=22) focus on the effect of intervention on clinical symptomology; 32.3% (N=51) report a mix of effect and usability / UX assessment; 25.3% (N=40) focus solely on usability / UX.

Figures and tables (in main body of the paper)

Table 1: Distribution of assessment type.

Distribution of assessment types.
Assessment type ALL - N (%) RCT - N (%) PILOT RCT - N (%)
NO ASSESSMENT 45/158 (28.5%)* N/A N/A
ASSESSMENT (total) 113/158 (71.5%)* 25/113 (22.1%)† 7/113 (6.2%)†
USABILITY/UX 40/113 (35.4%)† 2/25 (8%)‡ 0/7 (0%)‡
EFFECT AND USABILITY/UX 51/113 (45.1%)† 14/25 (56%)‡ 5/7 (71.4%)‡
EFFECT 22/113 (19.5%)† 9/25 (36%)‡ 2/7 (28.6%)‡
* Percentage based on the total number of studies (N=158)
Percentage based on the number of studies with an assessment (N=113)
Percentage based on the number of RCT (N=25) and Pilot RCT studies (N=7), respectively

Figure 2: Temporal trend and number of articles published and assessment type.

It is a stacked bar chart over years, and the value of each group (assessment type, coloured variable) is in absolute counts. Text within each bar includes the number of papers plus relative percentage, both referring to each group.

Figure 3: Distribution of articles per mental disorder, categorized according to assessment type. The embedded pie chart shows the proportional distribution of types of assessment over all mental disorders.

The main plot is a stacked barchart colored by assessment type along with a lollipop chart (white circle at the end of each bar) that shows the total percentatge of each mental disorder. Note that we draw two main colors in the legend: purple represents no assessment at all, the range of greens denotes distinct types of assessment. A pie chart is embedded to show the proportional distribution of types of assessment over all mental disorders.

To complement the previous figure, some percentages are discussed in the section ‘Relation between assessment types and mental disorder’ which are included in the following table (that’s not included in the paper).

Mental Disorder Type of assessment Studies (N) Studies (%)
Depressive disorders EFFECT 6 19.4%
EFFECT AND USABILITY/UX 12 38.7%
NO ASSESSMENT 9 29%
USABILITY/UX 4 12.9%
Various disorders EFFECT 2 9.5%
EFFECT AND USABILITY/UX 7 33.3%
NO ASSESSMENT 4 19%
USABILITY/UX 8 38.1%
Anxiety disorders EFFECT 5 27.8%
EFFECT AND USABILITY/UX 4 22.2%
NO ASSESSMENT 8 44.4%
USABILITY/UX 1 5.6%
Substance-related and addictive disorders EFFECT 2 12.5%
EFFECT AND USABILITY/UX 5 31.2%
NO ASSESSMENT 8 50%
USABILITY/UX 1 6.2%
Schizophrenia spectrum and other psychotic disorders EFFECT AND USABILITY/UX 4 26.7%
NO ASSESSMENT 5 33.3%
USABILITY/UX 6 40%
Trauma and stressor-related disorders EFFECT 2 13.3%
EFFECT AND USABILITY/UX 7 46.7%
NO ASSESSMENT 1 6.7%
USABILITY/UX 5 33.3%
Suicidal behavior disorder/nonsuicidal self-injury EFFECT 1 9.1%
EFFECT AND USABILITY/UX 5 45.5%
NO ASSESSMENT 3 27.3%
USABILITY/UX 2 18.2%
Comorbid disorders EFFECT 1 14.3%
NO ASSESSMENT 2 28.6%
USABILITY/UX 4 57.1%
Bipolar and related disorders EFFECT 1 16.7%
EFFECT AND USABILITY/UX 2 33.3%
USABILITY/UX 3 50%
Obsessive-Compulsive and related disorders EFFECT 1 20%
EFFECT AND USABILITY/UX 3 60%
USABILITY/UX 1 20%
Neurodevelopmental disorders EFFECT AND USABILITY/UX 1 25%
NO ASSESSMENT 1 25%
USABILITY/UX 2 50%
Feeding and eating disorders NO ASSESSMENT 1 25%
USABILITY/UX 3 75%
Sleep-wake disorders EFFECT 1 33.3%
NO ASSESSMENT 2 66.7%
Personality disorders EFFECT AND USABILITY/UX 1 100%
Major and Mild Neurocognitive Disorders NO ASSESSMENT 1 100%

Figure 4: Temporal distribution of articles published for the top 6 mental disorders.

A line chart to show the number of papers per mental disorder and year.

Table 2: Apps and studies grouped by mental disorder. NA = app name not available/not mentioned. Apps covered in multiple publications are put in italics.

A tabular, compact distribution of apps per mental disorder, grouping the references tha point the same app together. The number(s) in brackets next to the app name is the reference(s) in which the app is mentioned. In the table below, these references are internal identifiers instead.

Compact distribution of papers (apps) per mental disorder ‘NA’ is app not specified/available
Mental Disorder References by app
Major and Mild Neurocognitive Disorders Rico (452)
Personality disorders EMOTEO (106)
Sleep-wake disorders Sleepcare (181), SleepIO (094;103)
Feeding and eating disorders Jorvie (265), Student Bodies–Eating Disorders (SB-ED) (007), Recovery Record (092;460)
Neurodevelopmental disorders iCanLearn (125), LifePal (142), My MFG (304), TimeOut (454)
Obsessive-Compulsive and related disorders Geo-Feedback App (262), GGOC (473), Live OCD Free (044), Mayo Clinic Anxiety Coach (056), RAW HAND (448)
Bipolar and related disorders MyT (042), PRISM (048), SIMPLe (258;226;138;459)
Comorbid disorders CASA-CHESS (297), Enuresis Trainer (290), Learn To Quit (300), SMI-CM (232), Stay Quit Coach (463), Stop-Cannabis (097), NA (086)
Suicidal behavior disorder/nonsuicidal self-injury BackUp; mEMA (453), BeyondNow (409), BRITE (470), iBobbly (124), Virtual Hope Box (VHB) (026;443), NA (275;012), Blue Ice (245;022;030)
Trauma and stressor-related disorders RELAX (301), NA (302), Life Armor; PE Coach; Positive Activity Jackpot; Eventful; Tactical Breather; Virtual Hope Box; Daily Yoga; Simply Yoga (121;037), PE Coach (062;061;069;049), PTSD Coach (160;118;306;015;088;450;478)
Schizophrenia spectrum and other psychotic disorders iCOPE (090), MindFrame (471), movisenseXS (420), RealLife Exp (419), SlowMo (260), TechCare (273), Temstem (418), Actissist (305;440), FOCUS (089;120), Heal Your Mind (087;406), PRIME (117;408)
Substance-related and addictive disorders Drink Less (468), Fit&Sobber (411), Mind the Moment (171), S-Health (437), SEVA (127), SmartQuit (217), Smoke Mind (065), Social-Local-Mobile (430), A-CHESS (013;307), CET Application (014;021), Kick.it (402;413), Smart-T (038;039)
Anxiety disorders Agoraphobia Free; Stress Free (291), Ångesthjälpen (122), AnxietyCoach (192), CBT Assistant (057), Challenger (464), Lantern (025), PsychAssist (286), Public Speech Trainer (PST) (050), SmartCAT (425), GET.ON PAPP (091;105), NA (123;308;027;285;163;457;472)
Various disorders ACT Daily APP (110), FOCUS (414), Headspace (177), iBobbly (029), iCare-stress (433), IntelliCare Suite (131), MoodMission (427), MoodTrainer (185), myCompass (126), PeerTECH App (119), Pocket Skills (436), Sinasprite (431), SmartCAT (282), SPIRIT (140), The Moment (281), TODAY! (294), Wellframe (034), WellWave (477), NA (475), WorkingWell (220;434)
Depressive disorders 7Cups (082), Be Good to Yourself (466), BlueWatch (401), Dcombat (235), Get Happy Program (058), HeadGear (438), iCare Prevent (455), MedLink (152), Mobile Sensing and Support (183), MoodHacker (187), Moodivate (188), MyGamePlan (203), PRIME-D (279), Push-D (407), SocioEmpathy (421), SPSRS (115), SuperBetter (231), The Sound Advice (428), Thought Challenger (114), TODAC TODAC (020), Kokoro-App (035;263;063), NA (051;023;266;071;303;077;432;435)

To complement the previous table some percentages are discussed in the section ‘Assessment types in research studies’, as part of the Discussion section, which are included here for reference.

Out of N=158, there are 138 studies with app names. However, there are 108 unique apps which follow a long tail distribution, since 89 appear only once, while 19 apps appear in two or more studies (an app in two studies is the most frequent case (14); an app in 3 or more studies occurs in 5 cases. In summary, the percentage of apps that have been studied (reused) in multiple articles is 19 / 138 (13.8%).

License

This document is licensed under a Creative Commons Attribution 4.0 International License.

The code is licensed under the MIT License.

The data used is licensed under a Open Data Commons Attribution License.

Runtime environment description.

## - Session info ----------------------------------------------------------
##  setting  value                       
##  version  R version 3.6.1 (2019-07-05)
##  os       Windows 10 x64              
##  system   x86_64, mingw32             
##  ui       RTerm                       
##  language (EN)                        
##  collate  Spanish_Spain.1252          
##  ctype    Spanish_Spain.1252          
##  tz       Europe/Paris                
##  date     2020-02-10                  
## 
## - Packages --------------------------------------------------------------
##  ! package      * version date       lib source        
##    assertthat     0.2.1   2019-03-21 [1] CRAN (R 3.6.0)
##    backports      1.1.4   2019-04-10 [1] CRAN (R 3.6.0)
##    base         * 3.6.1   2019-07-05 [?] local         
##    broom          0.5.2   2019-04-07 [1] CRAN (R 3.6.0)
##    callr          3.3.0   2019-07-04 [1] CRAN (R 3.6.1)
##    cellranger     1.1.0   2016-07-27 [1] CRAN (R 3.6.0)
##    cli            1.1.0   2019-03-19 [1] CRAN (R 3.6.0)
##    colorspace     1.4-1   2019-03-18 [1] CRAN (R 3.6.0)
##  P compiler       3.6.1   2019-07-05 [2] local         
##    cowplot      * 1.0.0   2019-07-11 [1] CRAN (R 3.6.1)
##    crayon         1.3.4   2017-09-16 [1] CRAN (R 3.6.0)
##  P datasets     * 3.6.1   2019-07-05 [2] local         
##    desc           1.2.0   2018-05-01 [1] CRAN (R 3.6.1)
##    devtools       2.1.0   2019-07-06 [1] CRAN (R 3.6.1)
##    digest         0.6.19  2019-05-20 [1] CRAN (R 3.6.0)
##    dplyr        * 0.8.2   2019-06-29 [1] CRAN (R 3.6.0)
##    ellipsis       0.2.0   2019-06-20 [1] CRAN (R 3.6.0)
##    evaluate       0.14    2019-05-28 [1] CRAN (R 3.6.0)
##    forcats      * 0.4.0   2019-02-17 [1] CRAN (R 3.6.0)
##    fs             1.3.1   2019-05-06 [1] CRAN (R 3.6.0)
##    generics       0.0.2   2018-11-29 [1] CRAN (R 3.6.0)
##    ggplot2      * 3.2.1   2019-08-10 [1] CRAN (R 3.6.2)
##    glue           1.3.1   2019-03-12 [1] CRAN (R 3.6.0)
##  P graphics     * 3.6.1   2019-07-05 [2] local         
##  P grDevices    * 3.6.1   2019-07-05 [2] local         
##  P grid           3.6.1   2019-07-05 [2] local         
##    gtable         0.3.0   2019-03-25 [1] CRAN (R 3.6.0)
##    haven          2.1.0   2019-02-19 [1] CRAN (R 3.6.0)
##    here         * 0.1     2017-05-28 [1] CRAN (R 3.6.1)
##    highr          0.8     2019-03-20 [1] CRAN (R 3.6.0)
##    hms            0.4.2   2018-03-10 [1] CRAN (R 3.6.0)
##    htmltools      0.3.6   2017-04-28 [1] CRAN (R 3.6.0)
##    httr           1.4.0   2018-12-11 [1] CRAN (R 3.6.0)
##    jsonlite       1.6     2018-12-07 [1] CRAN (R 3.6.0)
##    kableExtra   * 1.1.0   2019-03-16 [1] CRAN (R 3.6.1)
##    knitr        * 1.23    2019-05-18 [1] CRAN (R 3.6.0)
##    labeling       0.3     2014-08-23 [1] CRAN (R 3.6.0)
##    lattice        0.20-38 2018-11-04 [2] CRAN (R 3.6.1)
##    lazyeval       0.2.2   2019-03-15 [1] CRAN (R 3.6.0)
##    lubridate      1.7.4   2018-04-11 [1] CRAN (R 3.6.0)
##    magrittr       1.5     2014-11-22 [1] CRAN (R 3.6.0)
##    memoise        1.1.0   2017-04-21 [1] CRAN (R 3.6.1)
##  P methods      * 3.6.1   2019-07-05 [2] local         
##    modelr         0.1.4   2019-02-18 [1] CRAN (R 3.6.0)
##    munsell        0.5.0   2018-06-12 [1] CRAN (R 3.6.0)
##    nlme           3.1-140 2019-05-12 [2] CRAN (R 3.6.1)
##    pillar         1.4.2   2019-06-29 [1] CRAN (R 3.6.0)
##    pkgbuild       1.0.3   2019-03-20 [1] CRAN (R 3.6.1)
##    pkgconfig      2.0.2   2018-08-16 [1] CRAN (R 3.6.0)
##    pkgload        1.0.2   2018-10-29 [1] CRAN (R 3.6.1)
##    prettyunits    1.0.2   2015-07-13 [1] CRAN (R 3.6.0)
##    processx       3.4.0   2019-07-03 [1] CRAN (R 3.6.1)
##    ps             1.3.0   2018-12-21 [1] CRAN (R 3.6.0)
##    purrr        * 0.3.2   2019-03-15 [1] CRAN (R 3.6.0)
##    R6             2.4.0   2019-02-14 [1] CRAN (R 3.6.0)
##    RColorBrewer   1.1-2   2014-12-07 [1] CRAN (R 3.6.0)
##    Rcpp           1.0.1   2019-03-17 [1] CRAN (R 3.6.0)
##    readr        * 1.3.1   2018-12-21 [1] CRAN (R 3.6.0)
##    readxl         1.3.1   2019-03-13 [1] CRAN (R 3.6.0)
##    remotes        2.1.0   2019-06-24 [1] CRAN (R 3.6.1)
##    rlang          0.4.0   2019-06-25 [1] CRAN (R 3.6.0)
##    rmarkdown      1.13    2019-05-22 [1] CRAN (R 3.6.0)
##    rprojroot      1.3-2   2018-01-03 [1] CRAN (R 3.6.0)
##    rstudioapi     0.10    2019-03-19 [1] CRAN (R 3.6.0)
##    rvest          0.3.4   2019-05-15 [1] CRAN (R 3.6.0)
##    scales       * 1.0.0   2018-08-09 [1] CRAN (R 3.6.0)
##    selectr        0.4-1   2018-04-06 [1] CRAN (R 3.6.0)
##    sessioninfo    1.1.1   2018-11-05 [1] CRAN (R 3.6.1)
##  P stats        * 3.6.1   2019-07-05 [2] local         
##    stringi        1.4.3   2019-03-12 [1] CRAN (R 3.6.0)
##    stringr      * 1.4.0   2019-02-10 [1] CRAN (R 3.6.0)
##    testthat       2.1.1   2019-04-23 [1] CRAN (R 3.6.1)
##    tibble       * 2.1.3   2019-06-06 [1] CRAN (R 3.6.0)
##    tidyr        * 0.8.3   2019-03-01 [1] CRAN (R 3.6.0)
##    tidyselect     0.2.5   2018-10-11 [1] CRAN (R 3.6.0)
##    tidyverse    * 1.2.1   2017-11-14 [1] CRAN (R 3.6.1)
##  P tools          3.6.1   2019-07-05 [2] local         
##    usethis        1.5.1   2019-07-04 [1] CRAN (R 3.6.1)
##  P utils        * 3.6.1   2019-07-05 [2] local         
##    viridisLite    0.3.0   2018-02-01 [1] CRAN (R 3.6.0)
##    webshot        0.5.1   2018-09-28 [1] CRAN (R 3.6.1)
##    withr          2.1.2   2018-03-15 [1] CRAN (R 3.6.0)
##    xfun           0.8     2019-06-25 [1] CRAN (R 3.6.0)
##    xml2           1.2.0   2018-01-24 [1] CRAN (R 3.6.0)
##    yaml           2.2.0   2018-07-25 [1] CRAN (R 3.6.0)
## 
## [1] C:/Users/Carlos Granell/Documents/R/win-library/3.6
## [2] C:/Program Files/R/R-3.6.1/library
## 
##  P -- Loaded and on-disk path mismatch.